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PROPERTIES  OF  A  CLASS  OF  RELIABILITY  GROWTH  PLANNING  MODELS 


1.  EXECUTIVE  SUMMARY 

DoD  Directive-Type  Memorandum  (DTM)  11-003-  Reliability  Analysis,  Planning, 
Tracking,  and  Reporting,  March  21,  2011  [1],  applies  to  all  major  DoD  developmental 
acquisition  programs.  This  DTM  requires  that  reliability  growth  curves  (RGC)  for  such 
programs  be  included  in  the  Systems  Engineering  Plan  (SEP)  at  Milestone  A,  and  be  updated  in 
the  Test  and  Evaluation  Master  Plan  (TEMP)  beginning  at  Milestone  B.  The  RGC  is  to  “reflect 
the  reliability  growth  strategy  and  be  employed  to  plan,  illustrate,  and  report  reliability  growth.” 
Additionally,  the  Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics  ,  and 
Technology  (ASA(ALT))  issued  a  Memorandum  dated  June  26,  2011  [2],  addressing  the  subject 
’’Improving  the  Reliability  of  U.S.  Army  Materiel  Systems.”  This  document  states  that 
“Program  Managers  (PMs)  of  all  Acquisition  Category  I  (ACAT  I)  systems  and  for  ACAT  II 
systems  where  the  sponsor  has  determined  reliability  to  be  an  attribute  of  operational  importance 
shall  place  reliability  growth  planning  curves  in  the  SEP,  TEMP,  and  Engineering  and 
Manufacturing  (EMD)  contracts  and  ensure  that  U.S.  Army  systems  are  resourced  to  accomplish 
this  requirement.”  The  ASA(ALT)  document  stipulates  that  “Reliability  growth  planning  is 
quantified  and  reflected  through  a  reliability  growth  planning  curve  using  the  Planning  Model 
based  on  Projection  Methodology  (PM2).”  The  document  also  states  “Where  warranted  by 
unique  system  characteristics,  the  Army  Test  and  Evaluation  Command  (ATEC),  in  consultation 
with  the  Project  Manager,  may  specify  an  alternative  reliability  growth  planning  method.” 

Due  to  the  focus  on  the  PM2  reliability  growth  planning  model  [3,4]  in  the  ASA(ALT) 
document,  it  has  become  widely  used.  Recently,  an  Army  organization  that  wished  to  utilize  an 
alternate  method  had  a  consultant  conduct  a  review  of  this  planning  model.  A  number  of 
properties  of  PM2  were  identified  by  the  organization  and  their  consultant  that  were  deemed  to 
be  in  conflict  with  their  engineering  experience.  These  model  properties  were  examined  by  the 
consultant  for  the  continuous  version  and  discrete  version  of  PM2.  The  continuous  version 
addresses  the  case  where  growth  test  duration  is  measured  on  a  continuous  scale  such  as 
simulated  mission  hours  or  miles.  The  discrete  version  applies  to  the  case  where  test  duration  is 
measured  in  a  discrete  scale  such  as  trials  or  rounds.  The  consultant  concluded  that  four  of  the 
five  properties  examined  for  the  continuous  version  “are  counterintuitive  and  are  not  properties 
typically  desired  to  be  in  a  reliability  growth  planning  model.”  For  the  discrete  version,  four 
properties  were  examined  that  closely  align  with  four  corresponding  continuous  model  properties 
examined.  The  consultant  again  concluded  that  three  of  these  discrete  properties  “are 
counterintuitive  and  are  not  properties  typically  desired  to  be  in  a  reliability  growth  planning 
model.”  Since  the  PM2  discrete  version  properties  examined  mimic  the  corresponding 
continuous  version  properties,  only  the  continuous  version  of  PM2  is  addressed  in  this  paper. 

These  properties  for  the  PM2-Continuous  Model  will  be  examined  in  this  report  for  a 
large  class  of  problem  failure  mode  expected  rate  of  occurrence  functions,  which  includes  the 
PM2  function.  As  noted  in  Section  3,  this  large  class  of  functions  includes  the  problem  mode 
expected  rate  of  occurrence  functions  for  a  family  of  growth  models  advocated  by  the  late 
Professor  Miller  [5]  for  use  in  modeling  the  occurrence  of  new  software  bugs  in  reliability 
growth  testing.  It  will  be  demonstrated  that  the  “counterintuitive”  properties  noted  in  the  review 
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of  PM2  are  shared  by  reliability  growth  planning  models  that  are  based  on  this  large  class  of 
problem  failure  mode  rate  of  occurrence  functions.  Furthermore,  it  will  be  shown  that  the 
review’s  label  of  “counterintuitive”  for  the  first  three  examined  properties  is  based  on  sensitivity 
analyses  that  induce  tradeoffs  between  growth  parameters.  It  will  be  demonstrated  that  the 
induced  tradeoffs  do  not  conform  to  the  conditions  under  which  a  reliability  growth  test  is 
conducted.  The  last  “counterintuitive”  property  examined  is  shown  to  follow  from  general 
guidance  given  in  a  PM2  software  program  regarding  a  planning  risk.  Adhering  to  the  guidance 
yields  a  constraint  associated  with  the  initial  failure  rate.  However,  following  the  guidance 
would  lead  to  exactly  the  same  constraint  for  a  large  class  of  growth  models  that  includes  PM2. 
This  class  of  planning  models  is  even  broader  than  the  class  mentioned  above  and  further 
considered  in  Sections  3  through  6  with  regard  to  discussing  the  first  three  “counterintuitive” 
properties.  In  fact,  as  shown  in  Section  7,  the  resulting  associated  “counterintuitive”  property  of 
PM2  that  occurs  if  one  decides  to  adhere  to  this  risk  criterion  is  shared  by  every  reliability 
growth  planning  model  that  utilizes  a  function  that  is  a  suitable  candidate  for  portraying  the  rate 
of  occurrence  of  problem  failure  modes. 
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2. 


INTRODUCTION 


Reliability  Growth  (RG)  planning  is  an  area  of  reliability  growth  that  addresses  program 
schedules,  amount  of  testing,  resources  available,  and  the  realism  of  the  test  program  in 
achieving  its  goals.  Reliability  growth  planning  is  quantified  and  reflected  through  a  reliability 
growth  program  planning  curve.  Planning  curves  typically  express  a  measure  of  reliability  such 
as  Mean  Time  Between  Failure  (MTBF)  as  a  function  of  cumulative  test  duration  and  other 
programmatic  resources.  The  measure  of  test  duration  for  MTBF  is  test  simulated  mission  hours. 
A  RG  planning  curve  serves  as  a  baseline  against  which  reliability  assessments  may  be  compared 
throughout  the  test  program.  The  key  features  of  a  planning  curve  include  the  following:  (1)  The 
initial  MTBF,  Mi,  and  the  goal  MTBF,  Mg;  (2)  Test  phases  with  MTBF  steps  over  each  test 
phase,  where  the  step  represents  the  benchmark  MTBF  for  the  test  phase;  (3)  Corrective  Action 
Periods  (CAPs)  which  are  scheduled  calendar  periods  between  test  phases  for  implementing 
corrective  actions  (termed  fixes).  Most  growth  often  occurs  due  to  corrective  actions  in  the 
CAPs,  although  some  fixes  may  occur  in  the  test  phases;  (4)  An  idealized  MTBF  curve  from 
which  the  benchmark  MTBF  steps  over  the  test  phases  are  obtained.  A  sample  reliability  growth 
curve  and  its  features  are  illustrated  in  Figure  1. 


Test  Time  (hours) 

Figure  1.  Reliability  Growth  Planning  Curve 


The  idealized  MTBF  curve  at  cumulative  test  time  t  represents  the  MTBF  that  can  be 
obtained  if  all  the  problem  failure  modes  discovered  in  test  by  time  t  have  corrective  actions 
implemented  by  t.  The  steps  are  drawn  underneath  the  idealized  curve  to  reflect  an  average  lag 
time  from  when  a  problem  failure  mode  is  first  observed  to  when  a  corrective  action  can  be 
physically  implemented.  More  details  can  be  found  in  [3,4].  To  obtain  achievable  MTBF  steps, 
the  idealized  MTBF  curve  should  be  expressed  in  terms  of  achievable  input  planning  parameters 
that  are  under  the  control  of  the  Program  Office.  PM2  utilizes  such  planning  parameters. 
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The  key  planning  parameters  in  the  PM2  model  are:  (1)  The  initial  MTBF  goal  to  be 
obtained  by  scheduled  and  funded  design  for  reliability  activities  prior  to  the  growth  test 
developmental  period;  (2)  The  goal  developmental  test  (DT)  MTBF  that  supports  achieving  a 
successful  demonstration  of  the  MTBF  requirement  with  a  measure  of  assurance  (e.g.  statistical 
confidence);  (3)  The  developmental  test  time,  T,  over  which  problem  failure  modes  are 
discovered  that  are  expected  to  be  addressed  and  implemented  prior  to  or  during  the  last  DT 
CAP;  (4)  The  planned  management  strategy,  MS,  defined  to  be  the  expected  fraction  of  the 
initial  failure  intensity  due  to  failure  modes  that  will  be  addressed  by  corrective  actions  if  seen 
during  the  DT.  Such  modes  are  termed  B-modes  [3]  and  are  the  modes  that  were  previously 
referred  to  as  problem  modes.  The  remaining  failure  modes  are  termed  A-modes  [3];  and  (5)  The 
average  fix  effectiveness  factor  expected  to  be  achieved  during  the  DT  program.  For  an 
individual  B-mode,  the  fraction  reduction  in  the  initial  rate  of  occurrence  of  failures  due  to  the 
mode  as  a  result  of  a  fix  is  called  the  mode’s  fix  effectiveness  factor  (FEF).  The  expected 
average  of  these  mode  FEFs  is  the  planning  parameter  referred  to  in  (5)  above. 

Section  3  provides  several  key  reliability  growth  planning  concepts  associated  with  the 
idealized  MTBF  growth  curve.  Also  the  assumed  properties  of  the  B-mode  rate  of  occurrence 
function,  denoted  by  h(t),  are  stated  and  motivated.  In  Sections  4  -  7  it  is  shown  how  replacing 
the  h(t)  utilized  in  PM2  by  any  h(t)  in  the  class  of  rate  of  occurrence  functions  that  satisfy  the 
assumed  properties,  which  includes  the  PM2  h(t),  gives  rise  to  a  reliability  growth  model  that  has 
the  same  four  properties  deemed  counterintuitive  in  the  review  and  gives  rise  to  the  three  areas  of 
concern  noted  by  the  Army  organization.  Section  8  contains  remarks  about  the  addressed 
properties  based  on  the  facts  demonstrated  in  the  report. 
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3. 


BACKGROUND 


The  MTBF  idealized  curve  mentioned  above  is  generated  by  considering  the  two  types  of 
failure  modes  defined  in  Section  2,  i.e.,  the  A-modes  and  B-modes.  The  MTBF  idealized  curve 
is  generated  through  the  following  equation  for  its  reciprocal,  the  rate  of  occurrence  of  all 
failures  (termed  the  failure  intensity) 

p(t)  =  XA  +  (l  —  pd){XB  -  h(t)}  +  h(t)  (1) 

In  the  above  equation,  h(t)  is  the  rate  of  occurrence  of  B-modes  at  time  t.  It  is  also  equal 
to  the  expected  failure  intensity  due  to  the  B-modes  not  surfaced  by  time  t.  Using  this 
interpretation  of  h(t),  the  last  term  is  the  contribution  to  the  failure  intensity  due  to  these 
unobserved  B-modes.  In  this  equation  equals  h(0),  the  expected  failure  intensity  due  to  all  the 
B-modes  at  the  start  of  the  first  test  phase.  Thus  ab  -  h(t)  represents  the  expected  failure 
intensity  due  to  the  B-modes  that  were  surfaced  by  time  t.  In  the  second  term,  pd  denotes  the 
average  planned  fix  effectiveness  factor.  Thus,  the  second  term  represents  the  failure  intensity 
contribution  due  to  the  B-modes  surfaced  by  time  t  after  they  have  been  corrected  with  the 
planned  fix  effectiveness  factor.  Finally,  the  first  term  represents  the  failure  intensity 
contribution  due  to  the  A-modes. 

An  important  concept  associated  with  reliability  growth  planning  is  the  MTBF  growth 
potential  and  its  reciprocal,  the  failure  intensity  growth  potential.  These  are  denoted  by  Mgp  and 
Pgp,  respectively.  The  value  pgp  is  approached  in  the  limit  as  h(t)  goes  to  zero.  Thus,  Mgp  can  be 
viewed  as  a  ceiling  on  the  MTBF  that  can  be  achieved  for  a  given  design.  By  Equation  (1),  pGp 
and  its  reciprocal  Mgp  are  given  as  follows: 


,  .  Mt 

Pgp  —  +  (l  —  !ld)^B  -  MGP  —  —  —  (2) 

These  equations  and  concepts  not  only  apply  to  the  PM2  planning  model,  but  are  equally 
valid  for  any  reliability  growth  planning  model  that  utilizes  an  appropriate  form  for  h(t).  For 
PM2,  h(t)  is  given  by 


h(t)  = 


1  +  pt 


(3) 


where  P  is  a  positive  scale  parameter  for  h(t). 

In  Sections  4-7,  the  properties  of  PM2  that  were  deemed  counterintuitive  will  be 
discussed  for  a  class  of  growth  planning  models  that  are  generated  via  Equation  (1)  by  a  large  set 
of  h(t)  functions.  It  will  be  assumed  that  the  h(t)  in  this  class  of  functions  satisfies  a  set  of 
assumptions  that  include  the  fundamental  properties  advocated  by  Miller  [5].  Miller  deemed 
these  properties  appropriate  for  modeling  the  pattern  of  the  rate  of  occurrence  of  new  software 
bugs  for  reliability  growth  models.  These  fundamental  properties  can  be  motivated  by  first 
considering  the  exact  expression  for  the  expected  failure  intensity  due  to  the  B-modes  not 
surfaced  by  t,  denoted  hex(t).  To  do  so,  let  k  be  the  number  of  B-modes  residing  in  the  system  at 
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the  start  of  the  test  period,  and  let  Xi  be  the  rate  of  occurrence  of  failures  due  to  B-mode  i  at  the 
start  of  the  test  period  prior  to  corrective  actions.  It  can  be  shown  [6] 

k 

hex  (0  =  ^  he~kt  (4) 

i=l 

It  is  assumed  that  at  least  one  of  the  'k,  is  positive. 

Note  the  following  fundamental  properties  of  hex(t):  (i)  hex(0)  is  a  finite  positive  number; 
(ii)  hex(t)  approaches  0  as  t  increases;  and  (iii)  (-l)nhex(n)(t)  >  0  for  all  nonnegative  integers  n, 
where  hex(n)(t)  denotes  hex(t)  for  n  =  0  and  the  n’th  derivative  with  respect  to  t  of  hex(t)  for 
positive  n.  A  function  that  has  property  (iii)  is  called  completely  monotone  [5].  Observe  hex(t) 
has  too  many  parameters  for  use  in  a  reliability  growth  planning  model.  Thus,  for  complex 
systems,  a  parsimonious  approximation  to  hex(t)  is  typically  utilized.  In  the  remainder  of  the 
paper  h(t)  will  denote  a  parsimonious  approximation  for  hex(t).  It  will  be  assumed  that  h(t) 
satisfies  the  fundamental  properties  (i)  through  (iii)  above.  It  will  also  be  assumed  that  (iv)  the 
parameters  for  h(t)  include  a  scale  parameter,  P  >  0,  and  expected  initial  B-mode  rate  of 
occurrence,  Ab  =  h(0).  Note,  by  definition,  P  is  a  scale  parameter  of  h(t)  if  and  only  if  there  exists 
a  function  g  for  which  g(Pt)  =  h(t)  for  all  positive  P  and  nonnegative  t  for  each  minimal  fixed  set 
of  parameters  that,  in  addition  to  scale  parameter  P,  define  h.  As  implied  above,  the  minimal 
parameter  set  need  not  be  limited  to  just  h(0)  and  the  scale  parameter.  However,  it  is  assumed  the 
minimal  set  of  parameters  that  define  h  includes  P  and  An,  and  can  be  specified  independently 
within  their  defined  parameter  domains,  in  the  absence  of  any  imposed  conditions  other  than 
satisfying  (i)  through  (iii).  It  is  interesting  to  observe  that  the  power  law  function,  which  has 
been  used  for  reliability  projection  [3],  h(t)  =  aco  t0)" 1  ,  where  A  >  0  and  0  <  co  <  1,  can  be 
reparameterized  in  terms  of  a  scale  parameter,  s  >  0  ,  and  co.  In  this  parameterization,  h(t)  = 
M(st)“'1  where  s  =  X" 1  l/n"co)1  .  Observe  there  is  a  one-to-one  correspondence  between  (s,co)  e  O  and 
( k,co )  e  O  where  O  =  {(x,y)IO  <  x  and  0  <  y  <  1 }.  Note  however,  the  power  law  h(t)  does  not 
satisfy  Property  (i).  Also  note  the  PM2  h(t)  function  given  in  Equation  (3)  is  parameterized  in 
terms  of  the  scale  parameter  P  >  0  and  Ab  =  h(0).  To  specify  the  next  assumption  for  h(t),  let  p(t) 

denote  the  expected  number  of  B -modes  surfaced  by  test  time  t.  Thus  p(t)  =  h(x)dx.  Observe 
h(t)-r{  p(t)/t}  <  1  for  all  t  >  0.  The  assumption  is  that  (v)  limt_>00[  h(t)  -r-  {  jn (t) / 1}]  <  1,  where 
the  parameters  that  define  h  are  held  constant  as  t— >oo.  By  Proposition  A.l  in  Appendix  A  it 
follows  that  this  limit  equals  0  when  lim^o,  p(t)  <  qo.  Also,  when  lim^o,  p(t)  =  oo  and  t{h(t)} 
is  bounded  above  by  a  finite  value  one  has  lim^^h^)  -r-  {  ju  (t) / 1}]  =  0.  For  the  h(t)  utilized  by 
PM2  the  latter  situation  occurs  and  hence  the  limit  of  the  ratio  h(t)-^{  ju(t)/t}  goes  to  0  as  t— >oo. 
For  the  power  law  h(t)  =  kco  t®'1 ,  one  has  lim^^  h(t)  -r-  {  jli (t) / 1}]  =  co.  Note  to  have  the  rate  of 
occurrence  of  B -modes  approach  zero  as  t  — *  oo  requires  0  <  co  <  1.  This  choice  of  h(t)  with  co  <  1 
satisfies  the  above  limit  assumption,  even  though  h(0)  is  not  finite.  The  final  assumption  is  that 
(vi)  h(t)  can  be  expressed  as  the  product  AB  { ho(t) } ,  where  ho(t)  does  not  depend  on  ab.  The  PM2 
h(t)  satisfies  this  as  well.  More  generally,  the  h(t)  function  for  each  member  of  the  family  of 
Gamma  Exponential  Order  Statistic  (EOS)  Models  considered  by  Miller  [5, Section  5]  satisfy 
Assumption  (vi)  as  well  as  Assumptions  (i)  through  (v)  above.  The  h(t)  function  for  this  family 
can  be  expressed  as  follows: 
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(5) 


h(t)  = 


j-B 

(1  +  pt)a+2 


where  a  >  -2.  Note  P  in  Equation  (5)  equals  the  reciprocal  of  the  P  parameter  in  [5]  and  the  a  in 
(5)  is  one  less  than  the  a  parameter  in  [5]. 


For  a  >  -1,  the  limit  of  the  expected  number  of  B -modes  as  t— >co  is  finite.  In  this  case  the 
model  is  referred  to  as  a  Pareto  Nonhomogeneous  Poisson  Process  (NHPP)  [5,  Section  5].  When 
a>-l,  if  k  =  AB  P  (a  +  1)  is  a  positive  integer,  then  h(t)  can  also  represent  the  expected  rate 
of  occurrence  function  for  B -modes  associated  with  an  Independent  and  Identically  Distributed 
Order  Statistic  (IIDOS)  process  [5]  .For  this  IIDOS  process  the  B-mode  initial  failure  rates  are 
viewed  as  the  realization  of  a  random  sample  of  size  k  drawn  from  a  gamma  distribution  with 
mean  equal  to  /?(«  +  1)  and  variance  /?2(a  +  1).  The  h(t)  function  for  this  case  corresponds  to 
the  h(t)  of  the  finite -k  AMSAA  Maturity  Projection  Model  (AMPM)  in  [6].  If  one  re¬ 
parameterizes  h(t)  in  (5)  in  tenns  of  ,  /?,  and  k  and  utilizes  the  domains  Ab  >  0,  P  >  0  and  k  a 
positive  integer  then,  with  respect  to  these  parameters  and  associated  domains,  h(t)  satisfies 
assumptions  (i)  through  (vi).  The  PM2  h(t)  function  corresponds  to  the  case  where  a  =  -1.  For 
this  case,  the  limit  of  the  expected  number  of  B-modes  as  t— *x  is  infinite.  This  form  for  h(t)  was 
utilized  by  Musa  and  Okumoto  [7]  as  the  rate  of  occurrence  of  software  failures  in  a  NHPP 
model.  For  -2  <  a  <  -1  the  expected  number  of  B-modes  is  also  infinite.  The  resulting  model  is 
termed  a  Generalized  Power  Law  NHPP  Model  [5].  In  [5,  Section  5],  the  power  law  p(t) 
function  is  shown  to  be  a  limit  of  p(t)  functions  that  belong  to  Generalized  Power  Law  NHPP 
Models.  From  (5),  one  can  show  p(t)  for  a  Generalized  Power  Law  NHPP  Model  is  as  follows: 

"(t)  =  {=(5Tw]f(1  +  w_<“+1)-1)  (6) 

To  obtain  the  power  law  expected  B-mode  function,  the  limit  of  ju(t)  in  (6)  is  taken  as  P 
approaches  oo.  To  avoid  degenerate  limits  that  correspond  to  either  ju(t)  =  oo  or  u(t)  =  0  for  all  t 
>  0,  the  limit  is  taken  subject  to  ju(to)  =  mo  where  to  >  0  and  mo  is  a  positive  integer.  These 
parameters  are  held  fixed,  along  with  a  e  (-2,-1),  as  P  — >  oo.  From  the  condition  that  m0  —  /u(t0) 
and  (6)  one  obtains 


— ( a  +  1  )/?m0 

{(i+ /?(„)-<«+») -i} 


Substituting  the  expression  for  XB  in  (7)  into  (6)  yields 


Kt)  = 


'  (1  +  /?t)~(a+1)  -  1 ' 
(1  +  [ltQ)-(a+1)  -  1 


(7) 


(8) 


To  emphasize  that  to,  mo  and  a  in  (8)  are  held  fixed  as  P  — >  oo,  the  jn(t)  in  (8)  will  be 
denoted  by  p(t;  t0,  m0,  a,  P).  Note 
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ju(t;  t0,m0,a,p )  =  m0 


(9) 


rl  +  /?t|  (a+1)  1  (i  +  /?t)-(a+D 

ll  +  /?t0)  J  i _ 1 _ 

l  (1  +  /?t0)~(a+1) 


Also  observe 


-2<a<-l<->-l<a+l<0<->0<  -(a  +  1)  <  1 


It  follows  from  (9)  and  (10)  that 


li(t)  t0,m0,a )  = 


where 


lim  /r(t;  t0, m0,  a, /?)  =  m0  (— ) 

f  \t0/ 


-(a+lp 


=  At6 


me, 

A  =  _(g+1 3  and  <u  =  -(a  +  l)e  (0, 1) 

to 

Setting  t0  =  1  ,  from  (12)  one  obtains 

A  =  m0  and  a>  —  —(a  +  1) 


(10) 


(11) 


(12) 


(13) 


Equations  (11),  (12)  and  (13)  show  that  each  power  law  mean  value  function  /r(t)  =  At" 
where  A  >  0  and  0  <  oo  <  1  can  be  obtained  as  the  limit  of  Generalized  Power  Law  NHPP 
Models  that  satisfy  Assumptions  (i)  through  (vi)  above.  Note  the  associated  power  law  expected 
B-mode  rate  of  occurrence  function  is  h(t)  =  Aro  t05'1  =  mo{  -(a  +  l)}t'(a+1)1  for  t  >  0  .  Observe  the 
power  law  itself  does  not  correspond  to  a  member  of  the  Generalized  Power  Law  NHPP  family 
since  h(t)  is  not  finite  at  the  origin.  It  is  interesting  to  note  that  the  Generalized  Power  Law 
Models  provide  examples  of  h(t)  functions  that,  unlike  the  power  law  h(t),  satisfy  all  the 
assumptions  (i)  through  (vi)  above  and,  for  Assumption  (v) ,  lim^^l^t)  -r-  {  p(t)/t}]  =  -(a  +  1) 
where  0  <  -(a  +  1  )  <  1.  Miller  conjectures  [5,  pg.59]  that  the  family  of  Gamma  Exponential 
Order  Statistic  Models  “may  be  rich  enough  to  represent  adequately  the  entire  family  of  EOS 
Models  (defined  in  [5]).”  With  respect  to  the  family  of  EOS  Models,  Miller  [5,  pg.59]  states 
“The  possible  patterns  of  underlying  failure  rates  are  unlimited.” 


To  obtain  the  associated  reliability  growth  model,  the  parsimonious  approximation,  h(t), 
for  the  expected  B-mode  rate  of  occurrence  function  is  utilized  in  Equation  (1).  The  resulting 
idealized  failure  intensity  would  be  used  with  test  phases,  specified  lag  times,  and  the  program 
test  schedule  to  obtain  the  test  phase  MTBF  benchmarks  and  associated  metrics  as  described  in 
MIL-HDBK  189-C  [3]  for  the  PM2  h(t)  function.  As  indicated  above,  the  PM2  h(t)  is  one 
member  of  the  large  class  of  h(t)  functions  that  satisfy  Assumptions  (i)  through  (vi)  above. 
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4. 


PROPERTY  1  -  EFFECT  OF  VARYING  TEST  TIME  FOR  FIXED  GOAL  MTBF 


The  first  property,  corresponding  to  the  first  topic  of  concern  of  the  Army  organization 
sponsor  of  the  PM2  review,  can  be  paraphrased  as  follows:  Every  growth  curve  will  achieve  the 
DT  goal  MTBF  at  time  T,  for  any  T.  This  statement  is  correct  from  the  point  of  view  that  as  long 
as  the  DT  goal  MTBF  is  less  than  the  growth  potential  MTBF,  expressed  in  Equation  (2)  in  terms 
of  input  parameters,  then  one  can  construct  a  growth  curve  that  goes  from  the  user  input  initial 
MTBF  to  the  goal  MTBF  in  the  specified  time  T  with  user  prescribed  values  for  the  management 
strategy  and  average  FEF.  This  was  one  of  the  model  properties  deemed  counterintuitive  by  the 
consultant  and  will  be  referred  to  as  Property  1  in  the  paper.  This  scenario  in  which  T  varied  and 
the  other  planning  parameters  were  held  fixed,  forces  a  tradeoff  between  T  and  the  model  scale 
parameter  p,  as  illustrated  in  Figure  2.  Here,  p  denotes  the  scale  parameter  for  h(t),  where  h(t)  is 
assumed  to  meet  the  assumptions  of  the  class  of  B-mode  rate  of  occurrence  functions  described 
in  Section  3. 


To  address  Property  1,  let  0(t)  be  defined  as  follows: 


0(f)  = 


^B-h(t)  h(t) 

h(0) 


(14) 


Observe  9(t)  is  the  expected  fraction  of  contributed  by  the  B-modes  discovered  in  test 
by  time  t.  Since  P  is  a  scale  parameter  for  h(t),  it  is  also  a  scale  parameter  for  9(t).  Thus  there 
exists  a  function  cp(x )  such  that  cp(pt)  =  0(t)  for  all  P  >  0  and  t  nonnegative.  Note  from  (14), 


h(t)  =  ^b(1  -  9(f))  =  ^b  { l-«POt)} 


From  the  definition  of  management  strategy  for  A|  =  (Mi)’1, 


(15) 
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MS  1  -  MS 

Xb  =  (MS)Xi=—  ,  kA  =  (1-MS)^  =  — 


(16) 


From  Equations  (1),  (15)  and  (16),  one  can  express  the  idealized  MTBF  at  time  T,  M(T), 
as  follows: 

i  -  (mswkPT)  (17) 


Mg  =  M(T)  = 


For  the  scenario  for  Property  1,  the  initial  and  DT  goal  MTBF  are  fixed,  as  is  the 
management  strategy  and  the  average  FEF.  Thus  as  T  is  varied,  (3  must  vary  such  that 

pT  =  C  (18) 

where  C  is  a  positive  constant  for  which  (17)  is  satisfied.  A  unique  such  constant  exists  since 


Mj  <  Mg  <  Mgp  (19) 

The  existence  and  uniqueness  of  such  a  constant  follows  from  the  facts  that  (i)  cp(x)  is  an 
increasing  continuous  function  over  [0,1);  (ii)  cp(0)=0;  and  (iii)  the  limit  of  cp(x)  equals  1  as 
x— >oo.  Thus  each  idealized  growth  curve,  corresponding  to  an  h(t)  from  the  considered  class  of 
B-mode  rate  of  occurrence  functions,  can  be  made  to  yield  a  specified  Mg  for  every  T  >  0, 
provided  Equation  (19)  holds.  Of  course  reducing  T  too  far  will  yield  an  unrealistically  large 
value  of  P  >  0  as  measured  by  risk  metrics  analogous  to  those  considered  in  MIE-HDBK-189C 
[3]  for  PM2.  One  risk  metric  is  based  on  the  following  fact:  Under  this  scenario,  one  can  show  as 
P  increases,  the  jump  from  the  first  MTBF  benchmark  step  to  the  second  MTBF  step  will  be  an 
increasing  percentage  of  the  total  growth  from  Mi  to  Mg.  Furthermore,  one  can  show  the 
expected  number  of  B-modes  that  contribute  to  this  jump  will  be  decreasing  as  P  increases. 

It  is  interesting  to  consider  the  original  MIF-HDBK-189  planning  model  [3]  whose 
model  equation  is  shown  below. 


M(T)  = 


(20) 


In  this  equation,  M(T)  denotes  the  DT  goal  MTBF  to  be  obtained  by  T.  Also  ti  denotes 
the  test  time  in  the  first  test  phase  and  Mi  denotes  the  average  MTBF  over  the  first  test  phase. 
Just  as  for  the  h(t)  scale  parameter  P,  for  a  specified  M(T),  the  smaller  T  is  the  larger  the  MIL- 
HDBK-189  growth  rate  a  will  be  in  Equation  (20)  (which  does  not  represent  the  parameter  a 
used  in  Section  3  for  the  Gamma  EOS  Model),  indicating  a  larger  average  rate  of  increase  in  the 
MTBF  over  the  test  period  of  length  T.  For  PM2,  the  relevant  concern  should  be  whether  the 
inputted  values  of  the  five  planning  parameters  result  in  a  value  of  the  scale  parameter  P  that  is 
realistic.  Does  this  resulting  value  of  P  conform  to  one’s  engineering  experience  and 
understanding  of  the  current  system  under  development?  This  is  comparable  to  asking  the 
following  with  respect  to  the  MIL-HDBK-189  Planning  Model:  Given  current  planning 
parameters  Mi,  ti,  T,  and  M(T)  in  Equation  (20),  is  the  associated  value  of  the  growth  rate  a 
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realistic?  In  the  case  of  the  MIL-HDBK-189  growth  rate,  one  could  compare  the  resulting 
growth  rate  to  historically  achieved  growth  rates  to  attempt  to  assess  the  risk  associated  with  the 
planning  parameters.  For  the  PM2  Model,  one  can  utilize  the  risk  metrics  in  [3],  as  well  as  any 
available  experience,  to  judge  the  plausibility  of  the  P  value  implied  by  the  input  planning 
parameters. 
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5. 


PROPERTY  2  -  TEST  TIME  EFFECT  ON  EXPECTED  NUMBER  OF  B-MODES 


The  second  property,  corresponding  to  an  area  of  concern,  can  be  expressed  as  follows: 

As  the  test  time  T  utilized  to  surface  problem  modes  that  contribute  to  growing  to  the  DT  goal 
MTBF  increases,  the  PM2  model  yields  the  same  DT  MTBF  goal.  Engineering  experience 
indicates  that  as  the  number  of  corrective  actions  increase  due  to  increasing  T,  the  resultant 
reliability  level  should  increase.  The  sensitivity  analsyis  conducted  in  the  review  for  this  property 
utilized  the  scenario  in  which  the  initial  MTBF,  DT  goal  MTBF,  management  strategy,  and 
average  FEF  were  held  fixed,  while  T  was  varied.  This  scenario  is  the  same  as  for  the  sponsor’s 
first  topic  of  concern  addressed  above  for  Property  1.  Such  a  scenario  does  not  pertain  to  the 
kind  of  engineering  experience  referenced  by  the  Army  organization  sponsor,  since  it  leads  to  a 
tradeoff  between  T  and  model  parameter  [3.  Changing  P  alters  the  assumed  distribution  of  the  B- 
mode  expected  initial  rates  of  occurrence  (termed  the  B-mode  profile)  arrived  at  by  design  for 
reliability  (DFR)  activities  conducted  prior  to  the  growth  test.  This  is  reflected  in  the  change  of 
the  average  expected  B-mode  failure  intensity  for  the  B -modes  surfaced  over  any  time  period 
[0,t0] -  To  see  this,  observe  by  Equation  (15),  the  average  expected  B-mode  failure  intensity  for 
the  B-modes  surfaced  over  any  time  period  [0,to]  is  given  by 


^"B,avgO-0’  P) 


-  h(tp)  _  Pcp(Pto) 

!l(t°)  f°{l  —  cp(x)}dx 


(21) 


where  p(t)  is  the  expected  number  of  B-modes  surfaced  by  t. 


However,  as  for  Property  1,  it  is  still  of  interest  to  see  whether  the  class  of  h(t)  functions 
with  a  scale  parameter  considered  in  Property  1  give  rise  to  Property  2  which  was  deemed 
counterintuitive  in  the  review  of  PM2.  To  do  so,  let  h(t;p)  and  p(t;P)  denote  h(t)  and  p(t), 
respectively,  with  scale  parameter  p. 

Also  let  T2  >  Tj  >  0.  Note  T,  under  the  postulated  scenario  forces  Pi  to  satisfy  Equation 
(18)  for  i  =  1,2.  Thus  0  <  P2  <  Pi  •  Recall  g(x)  is  decreasing  for  x  >  0  and  g(Pt)  =  h(t)  for  all 
P  >  0  and  t  >  0.  Thus,  as  illustrated  in  Figure  3, 

T  T  T 

m(T2;  P2)  =  f  h(t;  p2)dt  >  I  'h(t;  p2)dt  >  I  ‘h(t;  pjdt  =  (,(T,;  p,)  (22) 

J  o  20  2(5 
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Test  Period 

Figure  3.  Expected  Number  of  B-modes  Relationship 


This  shows  that  Property  2  holds  for  any  planning  growth  model  for  the  associated 
scenario  whose  h(t)  is  a  member  of  the  class  described  in  Section  3. 

Note  the  tradeoff  and  resulting  model  behavior  is  not  reflective  of  the  engineering 
experience  referenced  by  the  Army  organization  since  the  average  of  the  expected  surfaced  B- 
mode  failure  intensities  is  just  a  function  of  the  design  for  reliability  activities  accomplished 
prior  to  the  growth  test  and  thus  should  remain  constant  across  the  test  period.  Holding  (]  fixed 
along  with  the  planning  parameters  except  for  test  time  T  and  Mg  ,  one  can  inquire  whether  h(t) 
gives  rise  to  a  planning  model  that  conforms  to  the  referenced  engineering  experience.  Note  that 
for  this  modified  scenario,  the  corrective  actions  are  increasing.  To  see  this,  using  Equation  (15), 
one  has: 


(23) 


Observe  p(T)  is  increasing  via  Equation  (23)  since  Ab  and  P  are  fixed  and  T  is  increasing. 
Thus  the  number  of  corrective  actions  increases  as  T  increases.  This,  in  turn,  based  on  the 
referenced  engineering  experience,  should  lead  to  a  higher  achieved  value  of  Mq.  This  occurs  by 
Equation  (17)  since  cp(x)  is  an  increasing  function. 
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6. 


PROPERTY  3  -  EFFECT  OF  VARYING  Mi  FOR  A  SPECIFIED  GOAF  MTBF 


The  third  property  addressed  in  the  review  that  was  an  area  of  concern  can  be  expressed 
as  follows:  For  any  time  T,  increasing  Mi  increases  the  expected  number  of  corrective  actions  to 
achieve  the  DT  goal  MTBF  Mg-  The  engineering  expectation  of  the  Army  organization  was  that 
for  a  fixed  management  strategy,  average  FEF,  and  DT  goal  MTBF,  the  expected  number  of 
corrective  actions  to  achieve  Mg  should  decrease  as  the  initial  MTBF  is  raised  towards  Mg-  This 
is  a  reasonable  expectation  as  long  as  the  average  failure  intensity  associated  with  the  B -modes 
surfaced  over  the  test  period  T  remains  constant,  as  would  be  the  case  during  the  conduction  of  a 
system  reliability  growth  test.  However  in  the  sensitivity  analysis  conducted  in  the  review,  Mi  is 
varied  and  all  the  other  PM2  planning  inputs  (including  T)  remain  constant.  Thus,  by  Equation 
(17),  this  scenario  forces  [3  to  decrease  towards  zero  as  Mi  increases  towards  MG.  This  postulated 
scenario  is  not  reflective  of  engineering  experience  associated  with  executing  a  system  reliability 
growth  test.  As  shown  by  Equation  (21),  the  average  B-mode  failure  intensity  for  B-modes 
surfaced  during  the  test  period  from  0  to  T  is  just  a  function  of  P  and  does  not  change  over  the 
test  period.  However,  the  postulated  test  scenario  forces  P  to  change,  which  in  turn,  by  Equation 
(21),  would  lead  to  a  changing  average  B-mode  failure  intensity  for  the  test  period.  This 
postulated  scenario  in  which  a  forced  tradeoff  between  Mi  and  P  occurs  will  be  explored  in 
Section  6.2  for  the  class  of  h(t)  functions  discussed  in  Section  3  that  are  used  in  Equation  (1)  to 
generate  the  idealized  failure  intensity  and  associated  MTBF  step  reliability  growth  planning 
curve. 


6.1  Scenario  Pertinent  to  a  Reliability  Growth  Test  as  Mi  Varies.  First,  consider  an 
altered  scenario  that  corresponds  to  engineering  experience  associated  with  conducting  a 
reliability  growth  test  over  a  test  period  T.  In  this  scenario  the  management  strategy,  average 
FEF,  and  DT  goal  MTBF  Mq  shall  be  held  constant  as  the  initial  MTBF  is  raised  towards  Mg- 
To  have  the  scenario  be  pertinent  to  engineering  experience  associated  with  conducting  a 
reliability  growth  test,  the  average  expected  failure  intensity  of  B-modes  surfaced  over  any  test 
interval  is  fixed  and  determined  by  the  design  for  reliability  activities  prior  to  the  start  of  the  test. 
Thus  P  is  fixed.  However,  by  Equation  (17),  the  product  PT  must  be  decreasing  as  Mi  is 
increased  towards  Mg-  Thus  the  amount  of  test  time  T  required  to  obtain  the  DT  goal  MTBF  is 
decreasing,  in  accordance  with  engineering  experience.  Also,  since  T  and  AB  =  (MS)(f|)  are 
decreasing  and  P  is  fixed  ,  the  expected  number  of  B-modes  surfaced  by  T  is  decreasing  by 
Equation  (23).  This  in  turn  implies  that  the  expected  number  of  corrective  actions  required  to 
obtain  the  DT  goal  MTBF  should  be  decreasing  as  the  initial  MTBF  is  raised,  which  conforms  to 
engineering  experience. 

6.2  Effect  on  Expected  Number  of  B-modes  as  Mi/Mg  Varies.  Now  consider  the  scenario 
associated  with  Property  3,  i.e.,  all  planning  parameters  are  assumed  fixed  except  for  Mi.  This 
section  assumes  that  0  <  (MS)pd  <  1.  Contrary  to  what  Property  3  states  in  the  review,  for  this 
scenario  one  can  show  that  there  exists  r|i  and  rp  with  l-(MS)pd  <  Pi  <  y\2  <  1  that  satisfy  the 
following:  As  Mi  is  increased  towards  Mg,  the  expected  number  of  B-modes  by  T  is  an 

Mt 

increasing  function  of  n=  —  for  l-(MS)ud  <  n  <  ru  and,  when  (MS)ud  >  0.5,  a  decreasing 

Mg 

function  of  q  for  rp  <  q  <  1 . 
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In  the  following,  let  r|  =  f(y)  denote  the  function  that  relates  r|  to  y  =  pT,  where  P  >0  is 
the  scale  parameter  for  h(t)  and  T  is  fixed.  By  Equation  (17)  in  Section  4  one  has 

rr-  =  T1  =  f(y)  =  1  -  (MS)p  cp(y)  (24) 

mg 

Since  cp(y)  is  an  increasing  function  of  y  >  0,  q  is  a  decreasing  function  of  y  >  0-  By 
Equation  (24), 

f(0)  =  1  and  lim  f(y)  =  1  -  (MS)pd  (25) 


where  0  <  l-(MS)pd  <  1. 

Also  note  from  Equation  (17)  and  the  definition  of  y,  one  has  for  y  >  0, 

=  (MS*,  =  (MS)  [r _  (M'^,  |ip(y){  -d  p  =  yT-  (26) 

For  this  section,  let  p(T;y)  denote  the  expected  number  of  B-modes  surfaced  by  T  where 
7,b  and  P  are  given  by  the  equations  in  (26).  Next,  it  will  be  shown  that  there  exists  yi  and  y2  such 
that  0  <  y2  <  yi  <  oo  for  which  p(T;  y)  is  an  increasing  function  of  y  for  0  <  y  <  y2 ,  provided 
(MS)pd  >  0.5,  and  a  decreasing  function  of  y  for  yi  <  y.  Since  q  =  f(y)  and  f(y)  is  a  decreasing 
function  for  y  >  0,  it  will  follow  that  p(T)  is  an  increasing  function  of  q  for  l-(MS)pd  <  q  <  qi 
and  a  decreasing  function  of  q  for  q2  <  q  <  1 .  These  values  of  q  are  given  by  qi  =  ftyj  for  i  =1,2 
(see  Figure  4). 


Figure  4.  Relationship  Between  r\  =  Mi/Mg  and  y  =  PT 
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6.2.1  Effect  on  Expected  Number  of  B-modes  as  Mi/Mg  Approaches  One.  It  will 
now  be  shown  that  when  (MS)pd  >  0.5  there  exists  a  y2  >  0  such  that  p(T;y)  is  an  increasing 
function  of  y  for  0  <  y  <  y2.  By  Equations  (23)  and  (26)  one  has 


rT.  ,_f  (MS)PGT  u  1 

^  ,y  (l-  (MS)pdcp(y)j(  y 


fy  ) 

J  (p(x)dxj 


(27) 


Using  Equation  (27),  one  can  obtain  the  derivative  of  |a(T;y)  with  respect  to  y.  It  can  be 
shown  that 

dp(T;y) 


dy 


> 


r  i  ry  )  i  r  i  ry  ) 
0  ++  Q(y)  I1  —  —  J  (p(x)dxj  >  -  |cp(y)  -  -  J  (p(x)dxj 


(28) 


where 


Q(y)  = 


(MS)pd 


dcp(y) 

dy 


1  -  (MS)p.cp(y) 


(29) 


Note  the  limit  of  Q(y)  as  y  goes  to  zero  equals  (MS)pdL  where 


L  -  lim 

y^O 


dcp(y) 

dy 


(30) 


The  limit  in  Equation  (30)  exists  and  is  a  finite  positive  number  since  h(t)  satisfies  the 
fundamental  properties  (i)  through  (iii). 


Next  consider  the  limit  as  y  — >0  of  the  second  factor  on  the  left-hand  side  of  the  second 
inequality  in  (28).  This  can  be  shown  to  equal  to  1  by  applying  THopitaTs  Rule.  Applying 
THopitaTs  Rule  as  y— >0  to  the  right-hand  side  of  the  second  inequality  in  (28)  yields  (0.5)L. 
Thus  as  y  goes  to  zero,  the  limit  of  the  left-hand  side  of  the  second  inequality  in  (28)  is  greater 
than  the  corresponding  limit  of  the  right-hand  side  of  the  second  inequality  in  (28),  provided 
(MS)pd>  0.5. 


Therefore,  for  (MS)pd>  0.5,  there  exists  y2  such  that 


dp(T;y) 

dy 


>  0 


(31) 


for  0  <  y  <  yo  This  implies,  as  discussed  above,  u(T)  is  a  decreasing  function  of  —  =  n  =  f(y) 

mg 

for  q2  <  q  <  1,  where  q2  =  f(y2).  This  is  in  contradiction  to  the  corresponding  statement  of 
Property  3  in  the  review  of  PM2. 


6.2.2  Effect  on  Expected  Number  of  B-modes  as  Mi/Mg  Increases  Near  l-(MS)Pd- 

Next,  it  will  be  shown  there  exists  yi  >  y2  such  that  p(T;  y)  is  decreasing  for  y  >  yi.  In  the 
following  let  y  >  0.  By  (28)  one  has 
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db(T;y) 

dy 


< 


r  i  ry  )  1  r  1  ry  ) 

0  ~  Q(y)  H  -  -  J  (p(x)dxj  <  “My)  -  -  J  (p(x)dxj  (32) 


Note, 


1_~j  9(x)dx  =  j|y~  f  (p(x)dxj  >  0 


Thus  by  Equations  (32)  and  (33)  one  has 


dq(T;y) 


<  0  ~  Q(y)  < 


9(y)~/>(x)dx 


dy  y-/0ycp(x)dx 

The  second  inequality  in  (34)  can  be  written  as  follows: 

,  .  y(p(y)  -  Xf  cp(x)dx 
yQ(y)  < - -M-  - — 

y  -  Jo  (p(x)dx 

Inequality  (35)  is  equivalent  to  the  inequality  below: 

1-y<P(y)-/>(x)dx<i_yQ(y) 

y-  /0  (p(x)dx 


Note 


(33) 


(34) 


(35) 


(36) 


y<p(y)  ~  Ip  9(x)dx  _y~  Ip  (p(x)dx  -  y<p(y)  +  /0y  cp(x)dx  _  y{i  -  cp(y)) 
y  -  lo  (p(x)dx  y  -  /ycp(x)dx  /0y (i  -  cp(x)dx> 


From  (37)  and  the  equivalence  of  the  inequalities  in  (34),  (35)  and  (36)  one  obtains  the 
following  for  y  >  0: 


dq(T;  y) 


dy 


y(i  -  <p(y)} 

Sq{  1  -  (p(x)dx) 


<  i  -  yQ(y) 


(38) 


By  (29), 

yQ(y)  “  1  -  (MS)n„<p(y) 


(39) 


Note  cp(y)  is  a  nonnegative,  increasing,  bounded,  and  twice  differentiable  concave  function. 
Thus  by  Proposition  A.l  in  Appendix  A  one  has 
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(40) 


lim  y 

y->° o 


dcp(y) 

dy 


=  0 


Also,  liniy^oo  cp(y)  =  1  .  Thus  the  right  hand  side  in  the  second  inequality  in  (38),  by  (39)  and 
(40),  satisfies 


lim{l  -  yQ(y)}  =  1 

y->°° 


(41) 


To  consider  the  limit  of 
will  first  be  established. 


y{i-cp(y)} 

J0y{i-<p(x)dx} 


as  y— »co  in  the  second  inequality  in  (38),  some  notation 


Let  z  >  0.  It  will  be  convenient  to  express  h(z)  by  h(z;  pi,  p2)  where  pi  and  p2  are  positive 
parameters  that  are  held  fixed  as  z  — >  oo  .  Also,  it  is  assumed  that  any  additional  minimal  set  of 
parameters  that  are  required  to  define  h  are  kept  fix.  With  regard  to  pi  and  p2,  this  notation  will 
imply  that  pi  is  a  scale  parameter  for  h(z;  pi,  p2)  and  p2  =  h(0;  pi,  p2) .  Also  define 
0(z;  pi ,  p2)  =  {h(0;  pi,  p2)  -  h(z;  pi,  p2)}  4-  h(0;  pi,  p2).  Note  0(0;  ph  p2)  =  0.  Also,  by  the 
assumptions  imposed  on  h  and  the  definition  of  0,  it  follows  that  the  values  of  0  do  not  depend 
on  the  value  of  p2.  Thus,  the  value  0(z;  pi,  p2)  will  simply  be  expressed  as  0(z;  pi).  One  can  show 
that  pi  is  a  scale  parameter  for  0(z;  pi)  since  pi  is  a  scale  parameter  for  h(z;  pi,  p2).  More 
precisely,  it  can  shown  there  exists  a  function  cp(x)  such  that  cp(Piz)  =  0(z;  pi)  for  all  z  >  0  and 
pi  >  0  ,  for  each  fixed  minimal  set  of  parameters  that  include  pi,  p2  and  define  h. 


Recall  by  the  definition  of  y  in  Section  (6.2),  one  has  y  =  (3T.  Thus 

y{!  -  <p(y)}  PT{1  -  cp(pT))  p{i  -  cp(Tp)} 

Zoy{l  -  (p(x)dx)  /(|1T{1  -  (p(x)dx}  (i)  /0T(3{1  _  (p(x)dx) 

Observe,  by  (15), 

1  -  cp(Tp)  =  1  -  0(P;  T)  =  h(P;  T,  1) 

Let  z  =  x^T.  Then 

rTP  r P  rP 

I  (1  —  (p(x)dx)  =  T  I  (1  —  (p(Tz)}dz  =  T  I  (1  —  0(z;T)}dz 
Jo  Jo  Jo 

Thus 

fP 

T  (1  -  0(z;  T)}dz 


(f)/0  a-<pcx)dx>=(4) 


(42) 


(43) 


(44) 


(45) 


Equations  (15),  (42),  (43),  and  (45)  yield 
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(46) 


y{l  -<Ky)}  Ph(P;T,l)  Ph(P;T,l)  h(P;T,l) 

Jo^1  _  <P(x)dx)  /0P(1  -  0(z;  T)}dz  /0Ph(z;T,  l)dz  1) 

Equation  (46),  the  second  equation  in  (26)  and  the  previously  stated  assumption  that 

lim^fhCt)  -  {  jLx(t)/t}]  <  1 

when  the  parameters  defining  h  are  fixed  as  t— >oo  imply  the  following: 

y{i  —  <jP(y)}  h(p;  T,  1) 

/Qy{l  -  (p(x)dx}  0  fi(P<  Xi  1)  (^( 

Thus  Equations  (38),  (41),  and  (47)  demonstrate  there  exists  yi  >  y2  such  that  p(T;  y)  is 
decreasing  for  y  >  yi.  As  discussed  earlier,  this  implies  that  p(T)  is  an  increasing  function  of 

Mi 

—  =  n  =  f(y)  =  1  -  (MS)pd(p(y)  for  l-(MS)pd <r\<m,  where rp  =  f(yi). 
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7. 


PROPERTY  4  -  EFFECT  OF  LIMITING  THE  RATIO  OF  MG  TO  MGp 


The  last  of  the  four  “counterintuitive”  properties  in  the  review  can  be  stated  as  follows: 
Using  the  criterion  that  an  acceptable  growth  plan  should  have  the  DT  goal  MTBF  no  higher 
than  80%  of  the  growth  potential  MTBF,  the  PM2  Model  limits  the  use  of  commercial-off-the- 
shelf  items  (COTS)  from  a  low  of  0%  to  a  maximum  of  70%  of  the  initial  system  failure  rate.  In 
the  review  the  low  was  actually  mistakenly  stated  to  be  38%  which  is  the  percent  of  the  initial 
MTBF  to  the  goal  MTBF  that  limits  the  use  of  COTS  to  approximately  zero  percent  of  the  initial 
system  failure  rate.  Also,  the  DT  goal  MTBF  was  taken  to  be  the  Initial  Operational  Test  (IOT) 
MTBF  goal.  The  IOT  is  a  demonstration  test  conducted  by  service  personnel  in  simulated 
mission  scenarios  after  the  conclusion  of  the  DT  test  program  (see  Figure  1).  This  IOT  goal 
needs  to  be  set  sufficiently  above  the  requirement  MTBF  to  be  demonstrated  during  the  IOT, 

Mr,  to  have  a  reasonable  chance  of  being  able  to  successfully  demonstrate  the  requirement  with 
statistical  confidence  during  the  IOT.  Additionally,  the  DT  MTBF  goal  is  set  higher  than  the  IOT 
MTBF  goal,  Mg,iot,  if  a  drop  y  in  reliability  from  the  DT  to  the  IOT  environment  is  expected  due 
to  more  realistic  testing,  i.e.,  Mg.iot=  (l-y)MG.  Property  4  in  the  review  was  stated  for  y  =  0.  The 
PM2  software  characterizes  as  high  risk  a  ratio  of  the  DT  goal  to  the  growth  potential  greater 
than  0.80.  However  this  is  general  guidance,  independent  of  the  particular  growth  model.  The 
fact  that  the  COTS  percent  contribution  to  the  initial  system  failure  rate  must  be  less  than  71% 
(not  70%  as  stated  in  the  review)  under  the  0.80  guidance  is  independent  of  the  PM2  model.  It  is 
shown  below  this  guidance  results  in  the  same  71%  COTS  restriction  for  a  wide  class  of  growth 
planning  models. 


The  allowable  COTS  contribution  also  depends  on  the  assumed  drop  y  and  value  of  pd- 
Although  not  stated,  it  appears  that  the  review  set  pd  =  0.70,  and  y  =  0.0,  and  assumed  the  failure 
modes  associated  with  COTS  items  were  all  A-modes  and  that  these  modes  were  the  only  A- 
modes.  In  general,  neither  of  these  assumptions  need  be  true.  Under  these  assumptions,  the 
review  results  were  closely  reproduced  by  use  of  Equation  (49)  below.  For  this  case,  the  portion 

of  the  initial  system  failure  rate  due  to  COTS  is  -^  =  (1-MS).  This  ratio  can  be  calculated  as  a 

M 

Mr 

function  of - ,  by  solving  Equation  (48)  for  MS: 

mg, iot 


MG  _  Mg  _l-(MS)p 

mgp  [  Mi  ]  Mj  G 
U  -  (MS)pdj 

Utilizing  the  expression  for  MS,  one  obtains 


(48) 


(49) 


The  only  assumptions  about  the  reliability  growth  model  used  to  obtain  Equation  (49)  are 
the  following:  (a)  The  idealized  failure  intensity  associated  with  the  reliability  growth  model  is 
given  by  Equation  (1);  and  (b)  the  B-mode  rate  of  occurrence  function  in  Equation  (1)  is  a 
positive  decreasing  function  of  t  >  0  with  a  finite  initial  value  Ar  =  h(0).  Also  h(t) — >0  as  t— >oo. 
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.  From  the 


The  review  did  not  state  the  equation  utilized  to  calculate  —  as  a  function  of  Ml 

A]  Mg,IOT 

above,  it  is  apparent  that  the  “counterintuitive14  property  concerning  COTS  ascribed  to  PM2  is 
shared  by  a  large  class  of  growth  models  whose  h(t)  function  satisfies  the  basic  properties 
described  in  (b)  above. 
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8. 


REMARKS 


The  four  properties  of  PM2-Continuous  identified  in  a  review  as  being  counterintuitive 
were  addressed  in  the  paper.  It  was  demonstrated  that  these  properties  are  shared  by  reliability 
growth  planning  models  that  are  based  on  a  large  class  of  B-mode  rate  of  occurrence  functions 
h(t)  that  were  advocated  by  Miller  [5]  for  use  in  modeling  the  occurrence  of  new  software  bugs 
in  reliability  growth  testing.  The  label  of  “counterintuitive”  is  based  on  a  misapplication  of  the 
engineering  experience  associated  with  conducting  a  reliability  growth  test.  In  such  a  test  the  B- 
mode  initial  rates  of  occurrence  arrived  at  by  up-front  design  for  reliability  activities  are  fixed. 
However,  the  implied  model  parameter  trade-offs  associated  with  these  “counterintuitive” 
properties  change  the  implied  B-mode  profile  as  reflected  in  a  change  to  the  average  of  the 
expected  surfaced  B-mode  failure  intensities.  The  paper  demonstrated  that  if  the  scenarios 
associated  with  these  properties  are  modified  in  such  a  way  that  this  average  remains  constant, 
then  a  planning  model  generated  by  any  h(t)  in  the  class  described  conforms  to  the  engineering 
experience  referenced  by  the  review’s  sponsor. 
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APPENDIX  A  -  LIMIT  PROPOSITION  FOR  BOUNDED  CONCAVE  FUNCTIONS 


A-l 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK. 


A-2 


LIMIT  PROPOSITION  FOR  BOUNDED  CONCAVE  FUNCTIONS 


Proposition  A.l.  A  nonnegative,  increasing,  bounded,  and  twice  differentiable  concave  function 
vp(y)  defined  for  y  >  0  satisfies  the  following:  lim  y^oo  y  xp'(_y )  =  0. 

Proof  Let  y  >  0.  To  show  the  above  we  shall  utilize  a  function  w(y)  whose  slope  is  yi//(y). 

Thus  it  will  be  shown  that  limy^oo  w'(y)  =  0.  One  can  choose  w(y)  as  follows: 

ry 

w(y)  =  xi//(x)dx  (A.l) 

J  o 

Note  w'(y)  =  yi/;'(y).  A  useful  representation  for  w(y)  can  be  obtained  by  integration  by  parts. 

Observe  /  xxp'(x)dx  =  /  udv  =  uv  —  f  vdu  for  it  =  x  and  dv  —  i p'(x)dx.  Thus  du  =  dx 
and  one  can  choose  v  =  i/>(x).  Also  x  =  0  implies  u  =  0  and  x  =  y  implies  u  =  y.  From  the  above 
one  obtains  the  following: 

fy  fy  Cy 

I  xi//(x)dx  =  xi/i(x)  |  %  -  I  xp(x)dx  =  yxp(y)  —  I  ip(x)dx  (A.2) 

J0  J0  J0 

Since  0  <  i p(y)  is  increasing  and  bounded,  xp0 0  =  limy^OT  i p(y)  exists  and  is  a  finite  positive 
number.  From  the  concavity  of  v|/(x)  it  follows  that  i/f  (x)  is  decreasing.  Since  ip '  (x)  is 
decreasing  for  x  >  0  one  has  the  following  for  y  >  0  (see  Figure  5): 

0  <  yt/f  (y)  <  f  rp'(x)dx  =  xp(y )  <  ipm  <  oo  (A.3) 

Jo 


tx  0) 


y  x 

Figure  5.  Graphical  Depiction  of  (A.3) 
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Next  it  will  be  shown  that  limy^oo  yip'  (y)  exists.  Observe 


rV  rV 

ip(y)  —  I  ip'(x)dx  —  I  {ip' (x)  —  ip' (y)}dx  +  yip' (y) 
J  o  Jo 


Thus 


f 


iip'(x)  -  Ip'(y)}dx  =  Ipiy )  -  yip\y ) 


(A.4) 


(A.5) 


The  inequalities  in  (A. 3)  show  that  \|/(y)  is  bounded  above  by  ipm  and  that  yip'  (y)  lies  in  the 
finite  open  interval  (0,  ipm).  Thus  (A.5)  demonstrates  that  {ip'  (x)  —  ip'  (y)}dx  is  bounded 
above.  Also  this  integral  is  positive  for  y  >  0  since  ip' (x)  >  ip' (y)  for  0  <  x  <  y  (as  displayed 
in  Figure  5).  From  Figure  5,  one  can  see  that  this  integral  is  an  increasing  function  of  y  >  0.  This 
can  also  be  demonstrated  using  (A.5).  By  (A.5)  one  obtains  the  following: 


d  ry 

Tyl 


(x)  -ip'(y)}dx  =  ip' (y)  -{ ip'(y )  +yip"(y)}  =  -yip"(y )  >  0 


Since  f^{ip'  (x)  —  ip'  (y)}dx  has  been  shown  to  be  a  positive  increasing  bounded  function 

for  y  >  0,  it  follows  that  limy^oo  J^{ip'{x)  —  ip'(y)}dx  exists  and  is  finite.  Note  from  (A.4)  one 

obtains 


yip'iy)  =  ip(y)  -  f  {ip'W  -  ip'(y)}dx) 

Jo 


(A.6) 


Since  it  has  been  demonstrated  that  v|/(y)  and  f^{ip'(x)  —  ip'(y)jdx)  have  finite  limits  as  y  ->  oo, 
it  follows  from  (A.6)  that  limy^oo  yip'  (y)  exists.  Also  from  (A. 3)  one  has 

0  <  lim  yip'  (y)  <  ip <  oo 

y— >oo 


Let  s  =  lirm^oo  xip'  (x).  Suppose  s  >  0.  Then  there  exists  yx  >  0  such  that 

0  <  (0.5)s  <  xip'(x)  for  x  >  y1  (A.7) 


Let  y  >  y  1.  Then 
-y 


ry  ry i  ry  ry i  ry 

I  xip’(x)dx—  I  xip' (x)dx  +  I  xip' (x)dx  >  I  xip'(x)dx+  I  (0.5)sdx  (A.8) 

J 0  J 0  Jy1  Jo  Jy1 

Thus,  by  (A.8), 


rV  rVi 

I  xip'(x)dx>\  xip'(x)dx  +  (0.5)s(y  —  yq) 

J  o  J  o 


(A.9) 
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Therefore,  by  (A. 9),  for  y  >  y±  one  obtains 


y  1 1  xip'(x)dx  >y  1  j  xxp'(x)dx  +  (0.5)s  ^1  —  —  j  (A. 


10) 


Note 


l,m  jy-1  f  xVWdx  +  (0.5),  (l  - 1)}  =  t0. 5); 


(A.11) 


Thus,  by  (A.  10)  and  (A.l  1),  there  exists  y2  >  Ti  such  that  for  y  >  y2 


y  1  xxp'(x)dx  >  (0.4)s  >  0 
Jo 


(A.1 2) 


By  (A. 2)  one  has 


y 


rV  rV 

_1  I  xxp'(x)dx  —  ip(y)  —  y_1  I  ip(x)dx 
Jo  Jo 


(A.1 3) 


Note  liiriy^oo  xp{y)  =  ip^e  (0,  oo)  .  Therefore  lim^oo  Jy  xp(x)dx  —  oo.  Thus  by  l’Hopital’s 
Rule, 


=Bm 

y— »oo  y  y— >oo  U  y-  ~~ 


(A.  14) 


dy 


y 


It  follows  from  (A.13)  and  (A.14)  that 


rV  (  1p(x)dx'\ 

lim  y_1  xxp'{x)dx  —  lim  ip(y )  —  lim  j— - [  -  ip ^  —  xp ^  -  0  (A.15) 

JQ  y—>co  y^ooj  y  j 


y— >oo 


y—>oo 


y—>oo  | 


Observe  (A.15)  contradicts  (A. 12).  Thus  lim y-»oo  y  t/^Cy)  =  s  =  0. 
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